Fix pg_upgrade compatibility: omit removed catalog columns from views (re-release 0.2.2)#18
Merged
jnasbyupgrade merged 32 commits intoMay 1, 2026
Conversation
…test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`relhasoids` was removed in PG12. When binary pg_upgrade restores view definitions on PG12, `_cat_tools.pg_class_v` fails because it references `c.relhasoids` (included by the dynamic column-list builder on PG10/11 where the column exists). Explicitly add `relhasoids` to the omit list in `omit_column()` so the view is defined without it on all PostgreSQL versions, making binary pg_upgrade from PG10/PG11 to PG12+ work correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pg_upgrade writes multiple log files (pg_upgrade_internal.log, pg_upgrade_server.log, pg_upgrade_dump_*.log, etc.) to its working directory. Run it from a dedicated /tmp/pg_upgrade_logs dir and use `tail -n +1` on failure to print all logs with their filenames. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Pass --no-data-checksums to pg_createcluster so the new cluster matches the old cluster (which pg-start creates without checksums) - Use find+xargs instead of bare glob for log output so it doesn't fail when pg_upgrade exits before writing any log files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Only run on push to master (not PR branches); pull_request handles those, preventing each PR push from triggering two CI runs - Detect old cluster's data_checksums setting and pass --data-checksums to pg_createcluster only if needed, instead of --no-data-checksums which doesn't exist pre-PG18 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The psql query to detect checksums was running after pg_ctlcluster stop, so it always failed silently and checksum_flag was never set. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PG18 changed initdb's default to enable data checksums. Use pg_controldata (works on stopped clusters) to detect the old cluster's checksum setting, then: - If old cluster has checksums: pass --data-checksums to new initdb - If not, and new initdb supports --no-data-checksums (PG18+): explicitly disable to override the new default - Otherwise: no flag needed (pre-PG18 default is no checksums) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of detecting and matching checksums, recreate the old cluster with --data-checksums immediately after pg-start, and always pass --data-checksums when creating the new cluster. Both clusters consistently have checksums; no detection logic needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add pg_isready -t 30 after pg_ctlcluster start to wait for the socket to be available before proceeding - Remove the standard 'test' job (not needed on this branch) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pg_createcluster defaults to peer/md5 auth; the container runs as root so peer auth fails for user postgres. Pass --auth trust to match what pg-start configures. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Like relhasoids (removed in PG12), relhaspkey was removed in PG11. Without excluding it, binary pg_upgrade from PG10 to PG11+ fails with "column c.relhaspkey does not exist". Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Set PGUSER=postgres at job level, eliminating -U postgres / PGUSER=postgres throughout all steps - Set INITDB_OPTS=--data-checksums --auth trust at job level so both pg_createcluster calls stay in sync - Fix log capture on failure: PG17+ writes pg_upgrade logs to $new_datadir/pg_upgrade_output.d/ instead of CWD; search both paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
attcacheoff was an internal column in pg_attribute removed in PG18. Without excluding it, binary pg_upgrade to PG18 fails with "column a.attcacheoff does not exist". Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…_CONFIG - PGUSER: postgres moved to top-level env (applies to all jobs) - PGDATABASE set at step level where a specific db is needed - PG_CONFIG set at step level for new cluster install (make inherits env) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
README: warn that catalog-exposing objects are not a stable API Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove step-level env: blocks to avoid future duplication. PG_CONFIG is now at pg-upgrade-test job level alongside INITDB_OPTS. PGDATABASE values are inlined with -d flags since they differ between steps and a job-level value would affect make test calls. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
HISTORY: remove re-release note (now in PR description only) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ate test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…vel env Job-level PG_CONFIG broke the old cluster install since the new PG isn't installed yet at that point. Pass it inline as a make argument for the new cluster step instead. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ster Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
paths-ignore skips CI entirely when only *.md or *.asc files change. all-checks-passed is a single stable check name suitable for use as a required status check in branch protection rules. It fails if any job failed or was cancelled, passes if all passed or were skipped (e.g. on a docs-only push). It also self-inspects ci.yml to catch any jobs accidentally omitted from its needs list. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jnasbyupgrade
added a commit
that referenced
this pull request
May 1, 2026
… (re-release 0.2.2) (#18) Re-release of 0.2.2 (originally released 2026-04-27) fixing `pg_upgrade` compatibility with PostgreSQL 12 and 17+. `pg_upgrade` physically copies data files and re-applies schema definitions on the new cluster. Any view referencing a catalog column removed in the new PostgreSQL version will cause the upgrade to fail. The initial 0.2.2 release missed several such columns: - `relhasoids` — removed from `pg_catalog.pg_class` in PG12 - `relhaspkey` — removed from `pg_catalog.pg_class` in PG17 - `attcacheoff` — removed from `pg_catalog.pg_attribute` in PG17 Those columns are removed in this release. Also, the existing `pg-upgrade-test` CI job used `pg_upgradecluster` (dump/restore), which reinstalls extensions at the current version and masks exactly this class of problem. Replaced with binary `pg_upgrade`, which physically copies data files and correctly catches view/column incompatibilities across major versions. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
jnasbyupgrade
added a commit
to jnasbyupgrade/cat_tools
that referenced
this pull request
Jul 15, 2026
eb84bc6 Stamp 2.1.0 9fbe6f4 Fix results ordering, control file whitespace, ENABLE_* override, debug levels (Postgres-Extensions#31) 88bb4f2 Add Claude Code GitHub Actions workflows (Postgres-Extensions#41) af5bbbb ci: pass repo owners to run-tests.yml for fork-account matching (Postgres-Extensions#40) c7928af Fix repo-root guard to work inside a git worktree (Postgres-Extensions#39) b062fca ci: point reusable test workflow at @master b6cdbfd Add CI workflows and multi-session PR guard (Postgres-Extensions#33) 1ba0987 Stamp 2.0.3 1931cbe Fix pgxntool-sync remote and make it runnable without make (Postgres-Extensions#37) 8176304 Stamp 2.0.2 3e142ab Fix parse_control_file: remove comments before stripping quotes (Postgres-Extensions#27) cacc301 Stamp 2.0.1 bf1db6b Fix bash 3.2 / Linux compatibility issues (Postgres-Extensions#26) 62d0fcb Fix broken ifeq for --load-language=plpgsql on PG < 13 (Postgres-Extensions#24) 121f0b3 Stamp 2.0.0 ad3ca7e Remove .source support; add test/install, test/build, and verify-results (Postgres-Extensions#18) c010cf8 Fix bash 3.2 compatibility (Postgres-Extensions#23) abeb9d3 Remove .source file support from pg_regress integration (Postgres-Extensions#22) 08c1879 Stamp 1.1.2 6e0dad2 Fix double --dbname bug that defeated unique test database names 639756c Stamp 1.1.1 6ba3176 Fix pg_tle exception handler and empty upgrade files (Postgres-Extensions#15) 3b8cb2a Stamp 1.1.0 550a901 Remove commit.md (maintained in pgxntool-test) d73ca93 Add unique test database names to prevent conflicts (Postgres-Extensions#13) 9b344be Add update-setup-files.sh for 3-way merging after pgxntool-sync (#12) ab7f6e2 Stamp 1.0.0 3a571ba Add pg_tle support and modernize test infrastructure (#11) b96ea6d Add support for Claude code; build and doc improvements (#9) e9c24de Fix pg_regress on versions > 12 (#5) git-subtree-dir: pgxntool git-subtree-split: eb84bc6e87e21f2ced11ff1b8ddb4028b7c67c8f
jnasbyupgrade
added a commit
to jnasbyupgrade/cat_tools
that referenced
this pull request
Jul 15, 2026
…de scripts (fixes Postgres-Extensions#28) The machinery that generates our versioned SQL (the cat_tools--X.Y.Z.sql install scripts and cat_tools--A.B.C--X.Y.Z.sql update scripts) from .sql.in sources, and the DATA list that installs it, is moved out of the Makefile into a new, documented sql.mk (included after pgxntool/base.mk so it can use EXTENSION_VERSION_FILES, PG_CONFIG, MAJORVER, datadir, ...). sql.mk documents the GNU Make two-phase (parse vs. recipe) hazard at the root of Postgres-Extensions#28: we GENERATE the .sql we install, the generated files are gitignored and absent on a clean tree at parse time, and any parse-time $(wildcard) over them (including base.mk's DATA seed $(wildcard sql/*--*--*.sql)) silently comes up short -- the cached directory listing means it never notices them later either. The fix is to name-derive the install/update lists from the .sql.in SOURCES (which DO exist at parse time), feed the generated names into DATA explicitly, and $(sort) to dedup against base.mk's own wildcard. This makes the generated upgrade scripts land in DATA reliably on a clean build, so `make install` copies them and a later `ALTER EXTENSION cat_tools UPDATE` finds its path (Postgres-Extensions#28). The historical cat_tools--0.1.* install list is no longer hardcoded: it is derived by globbing the committed install-shaped .sql and subtracting the update scripts and everything we generate from .sql.in. Globbing COMMITTED files at parse time is safe; the rule of thumb is glob what git tracks, name-derive what we generate. Carry the Postgres-Extensions#18 pg_upgrade fix into the SOURCE. Postgres-Extensions#18 (re-release of 0.2.2) omitted catalog columns that later PostgreSQL versions removed -- relhasoids (PG12), relhaspkey (PG17), attcacheoff (PG17) -- so the stored view definitions do not reference since-removed columns and binary pg_upgrade succeeds. That fix was applied only to the GENERATED versioned files (sql/cat_tools--0.2.2.sql.in and the *--0.2.2 update scripts), never to the SOURCE sql/cat_tools.sql.in. That was harmless while the old Makefile deleted-and-did-not-reliably-regenerate the current-version file, but this refactor regenerates it in place from the source, which would REVERT the Postgres-Extensions#18 fix. Applying the same omit_column additions to sql/cat_tools.sql.in makes regeneration reproduce sql/cat_tools--0.2.2.sql.in byte-for-byte, preserving the released 0.2.2 SQL. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 15, 2026
Merged
jnasbyupgrade
added a commit
that referenced
this pull request
Jul 16, 2026
…#38) PR #18 (commit 31204e1, re-release 0.2.2) fixed binary `pg_upgrade` compatibility by omitting catalog columns that newer PostgreSQL removed from the `pg_class_v` / `pg_attribute_v` views: - `pg_class`: omit `oid`, `relhasoids` (removed PG12), `relhaspkey` (removed PG17) - `pg_attribute`: add `attcacheoff` (removed PG17) to the existing `oid`, `attmissingval` But #18 applied that fix **only to the generated versioned files** (`sql/cat_tools--0.2.2.sql.in` and the `*--0.2.2` update scripts), **never to the source** `sql/cat_tools.sql.in`. So running `make` regenerates the current-version file from source and **silently reverts** the #18 fix. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-release of 0.2.2 (originally released 2026-04-27) fixing
pg_upgradecompatibility with PostgreSQL 12 and 17+.
pg_upgradephysically copies data files and re-applies schema definitions onthe new cluster. Any view referencing a catalog column removed in the new
PostgreSQL version will cause the upgrade to fail. The initial 0.2.2 release
missed several such columns:
relhasoids— removed frompg_catalog.pg_classin PG12relhaspkey— removed frompg_catalog.pg_classin PG17attcacheoff— removed frompg_catalog.pg_attributein PG17Those columns are removed in this release.
Also, the existing
pg-upgrade-testCI job usedpg_upgradecluster(dump/restore),which reinstalls extensions at the current version and masks exactly this class
of problem. Replaced with binary
pg_upgrade, which physically copies datafiles and correctly catches view/column incompatibilities across major versions.